Consumers today are switching from film-based chemical photography to digital photography in increasing numbers. The instantaneous nature of image capture and review, the ease of use, numerous output and sharing options, multimedium capabilities, and on-line and digital medium storage capabilities have all contributed to consumer acceptance of this technological advancement. A hard drive, on-line account, or a DVD can store thousands of images, which are readily available for printing, transmitting, conversion to another format, conversion to another medium, or used to produce an image product. Since the popularity of digital photography is relatively new, the majority of images retained by a typical consumer usually takes the form of hardcopy medium. These legacy images can span decades of time and have a great deal of personal and emotional importance to the collection's owner. In fact, these images often increase in value to their owners over time. Thus, even images that were once not deemed good enough for display are now cherished. These images are often stored in boxes, albums, frames, or even their original photofinishing return envelopes.
Getting a large collection of legacy images into a digital form is often a formidable task for a typical consumer. The user is required to sort through hundreds of physical prints and place them in some relevant order, such as chronology or sorting by event. Typically, events are contained on the same roll of film or across several rolls of film processed in the same relative time frame. After sorting the prints, the user would be required to scan the medium to make a digital version of the image. Scanning hardcopy image medium such as photographic prints to obtain a digital record is well known. Many solutions currently exist to perform this function and are available at retail from imaging kiosks and digital minilabs and at home with “all-in-one” scanner/printers or with personal computers equipped with medium scanners. Some medium scanning devices include medium transport structure, simplifying the task of scanning hardcopy medium. Using any of these systems requires that the user spend time or expense converting the images into a digital form only to be left with the problem of providing some sort of organizational structure to the collection of digital files generated.
The prior art teaches sorting scanned hardcopy images by physical characteristics and also utilizing information/annotation from the front and back of the image. This teaching permits grouping images in a specific chronological sequence, which can be adequate for very large image collections. However, if the images are scanned and organized, but are not rotated correctly, they will be recorded to CD/DVD or some other suitable storage medium in the wrong orientation. This results in a less than ideal experience for the end user.
Accordingly, if additional metadata can be acquired from an image, there are several improvements to the image that can be made. For example, in addition to organization, metadata indicating that an image is black-and-white vs. color can be used to correct the orientation of the image.
Knowledge of image orientation permits the correct orientation of an image on an output display. Several algorithms exist for determining the orientation of images.
U.S. Pat. No. 5,642,443, to Goodwin et al., describes a method of considering an entire set of images in a consumer's film order to determine the orientation of an entire order. A statistical estimate of orientation is generated for each image in the set. A statistical estimate for the entire order is derived based upon the estimates for individual images in the set. Goodwin et al teach deriving relevant probabilities from spatial distributions of colors within the image. Goodwin et al must view an entire order of images rather than a single image. There are applications that only contain one image that Goodwin et al will be unable to correctly orient.
Also, U.S. Pat. No. 4,870,694, to Takeo describes a method of determining the orientation of an image that contains a representation of a human body. The position of the human is used as a clue to the orientation of the image. Takeo is primarily applicable to radiographic applications as used in hospitals or medical clinics. It is unlikely a broad-based consumer application, because it depends on certain constraints, such as requiring a human figure within the image.
Additionally, U.S. Pat. No. 6,011,585, Anderson, describes a method of determining image format and orientation based upon a sensor present in the camera at the time of image capture. However, if a sensor is not present in a particular camera or image-capturing device, the method of Anderson is not useful. The approach described by Anderson has the further disadvantage of requiring additional apparatus in the camera. Moreover, an image processing unit or operation will be unable to perform correct orientation unless the particular camera contained the additional apparatus. Likewise, this method is not able to find the orientation of a scanned photographic print because the state of the camera's sensor is not recorded on the photographic print.
Several other methods for determining the orientation of an image have been described where either low-level features (as described in U.S. Pat. No. 7,215,828) are extracted or objects are detected and used to determine the orientation of the image. For example, it is known to determine orientation of images based on looking for faces as discloses in U.S. Pat. No. 6,940,545 to Ray et al., but only about 75% of images contain faces and automatic face detectors sometimes miss detecting faces even when they are present, or find false faces that are not actually in an image. Other methods of determining image orientation are based on finding sky (see U.S. Pat. No. 6,512,846) or grass or street signs (as described in U.S. Pat. No. 7,215,828), but again many images do not contain these materials. Furthermore, the structure of lines and vanishing points in the image has been shown to be useful for determining the format and orientation of images (U.S. Pat. No. 6,591,005). Even considering all of these features, there are still many images that will not be oriented properly because they do not contain the sought after objects, or the object detectors were incorrect. Further complicating the problem of determining the orientation of scanned photographic prints is the fact that many prints contain no color information, which complicates and compromises the accuracy of the detection of sky and other materials.
In addition to the problem or properly orienting the images, for organizing and searching the image collection that contains scanned images, it is useful to know the image capture date of the images.